Image capture using target area illumination

ABSTRACT

Methods, systems, and apparatus include computer programs encoded on a computer-readable storage medium, including a method for processing images. A structure provides a light sources configured to illuminate a target area. Illumination is controlled in a sequence over a first time period. At a given point in time, light sources are powered to enable illumination of the target area. An image of a target object is captured from a single reference location. The captured image is evaluated including processing the evaluated images on a pixel by pixel basis to produce a single composite image having respective pixel values derived from the evaluated captured images. The target object is 3D scanned to determine height information for a surface of a target on the target area. The height information is used to flatten respective pixels of the composite image. The composite image is provided after the flattening of the respective pixels.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of, and claims priorityto, U.S. patent application Ser. No. 15/422,188, filed on Feb. 1, 2017,for 2D/3D Image Scanning and Compositing, now U.S. Pat. No. 9,786,080,which is a continuation application of U.S. patent application Ser. No.14/791,087, filed on Jul. 2, 2015, for 2D/3D Image Scanning andCompositing, now U.S. Pat. No. 9,563,957. The disclosures of theforegoing applications are incorporated here by reference.

BACKGROUND

This specification relates to image processing.

Images can be obtained from different sources, such as by obtaining(e.g., capturing) an image with a camera. Various ways can be used toobtain an image, such as using a desktop scanner, taking a photographusing a digital camera, or providing analog documents to a service thatcan scan the documents and provide digital images.

Image quality can depend on various factors. For example, the number,quality and angle of light sources can produce shadows that may produceless than desirable image quality. Further, features of a given subjectmay affect the quality (producing artifacts or distortion) of atwo-dimensional (2D) image that is ultimately produced.

SUMMARY

In general, one innovative aspect of the subject matter described inthis specification can be implemented in methods that include acomputer-implemented method for capturing images. The method includesproviding a structure including a plurality of light sources, each lightsource being disposed on the structure and being configured toilluminate a target area in proximity to the structure. The methodfurther includes controlling illumination of the plurality of lightsources in a sequence over a first time period, wherein at a given pointin time in the sequence, one or more of the plurality of light sourcesare powered to enable illumination of the target area by powered ones ofthe plurality of light sources. The method further includes capturing,from a single reference location, an image of a target object that islocated in the target area at each point in time in the sequence. Themethod further includes evaluating the captured images includingprocessing the evaluated images, for example, on a pixel by pixel basis,to produce a single composite image having respective pixel valuesderived from the evaluated captured images. In some implementations, themethod further includes 3D scanning the target object to determineheight information for a surface of a target on the target area for oneor more pixels associated with the composite image. The method furtherincludes using the height information to flatten respective pixels ofthe composite image. The method further includes providing, as anoutput, the composite image after the flattening of the respectivepixels.

These and other implementations can each optionally include one or moreof the following features. The method can further include calibratingthe illumination including backing out discontinuities in lighting ofthe target area. The method can further include inserting a page intothe target area, the page representing the target object. The page canbe included in an album and wherein inserting includes inserting thealbum turned to the page. The plurality of light sources can includefour or more spatially separated light sources. Evaluating the capturedimages can include evaluating for each pixel each captured image,determining one or more pixel values from the captured images to use inthe composite image and determining a composite image pixel value for arespective pixel based on the determined one or more pixel values.Evaluating can include for each pixel, filtering out a portion of thepixel values for one or more of the captured images and not using thefiltered out portion in computing the determined composite image pixelvalue. Filtering out can include filtering out one or more lightestand/or one or more darkest pixel values. Evaluating can include applyinga mathematical function to pixel values from captured images. Themathematical function can be selected from the group comprising amathematical mean or median. Evaluating the captured images can includenormalizing each captured image with respect to the calibration prior tofurther processing. 3D scanning can include determining a point cloudassociated with the target area including height information anddefining a Bezier curve to define a surface associated with the targetarea, and using the defined surface to develop the height information.The structure can be portable. Evaluating the captured images caninclude applying a filter to each captured image.

In general, another innovative aspect of the subject matter described inthis specification can be implemented in computer program products thatinclude a computer program product tangibly embodied in acomputer-readable storage device and comprising instructions. Theinstructions, when executed by one or more processors, cause theprocessor to: provide a plurality of light sources, each light sourcebeing disposed on a structure and being configured to illuminate atarget area in proximity to the structure; controlling illumination ofthe plurality of light sources in a sequence over a first time period,wherein at a given point in time in the sequence, one or more of theplurality of light sources are powered to enable illumination of thetarget area by powered ones of the plurality of light sources; capture,from a single reference location, an image of a target object that islocated in the target area at each point in time in the sequence;evaluate the captured images including processing the evaluated imageson a pixel by pixel basis to produce a single composite image havingrespective pixel values derived from the evaluated captured images;optionally 3D scan the target object to determine height information fora surface of a target on the target area for one or more pixelsassociated with the composite image including using the heightinformation to flatten respective pixels of the composite image; andprovide, as an output, the composite image.

In general, another innovative aspect of the subject matter described inthis specification can be implemented in systems, including a systemcomprising one or more processors and one or more memory elementsincluding instructions. The system can include a structure, a pluralityof light sources and means for powering the light sources. Theinstructions, when executed, cause the one or more processors to:illuminate a target area in proximity to the structure; controlillumination of the plurality of light sources in a sequence over afirst time period, wherein at a given point in time in the sequence, oneor more of the plurality of light sources are powered to enableillumination of the target area by powered ones of the plurality oflight sources; capture, from a single reference location, an image of atarget object that is located in the target area at each point in timein the sequence; evaluate the captured images including processing theevaluated images on a pixel by pixel basis to produce a single compositeimage having respective pixel values derived from the evaluated capturedimages; optionally 3D scan the target object to determine heightinformation for a surface of a target on the target area for one or morepixels associated with the composite image and use the heightinformation to flatten respective pixels of the composite image; andprovide, as an output, the composite image.

In general, another innovative aspect of the subject matter described inthis specification can be implemented in a computer-implemented methodfor capturing images. The method includes providing a structureincluding a plurality of light sources, each light source being disposedon the structure and being configured to illuminate a target area inproximity to the structure. The method further includes capturing one ormore calibration images of the target area including illuminating thetarget area with one or more of the plurality of light sources andcapturing the one or more calibration images coincident with theilluminating. The method further includes receiving an indication that atarget item has been placed in the target area and thereaftercontrolling illumination of the plurality of light sources in a sequenceover a first time period, wherein at a given point in time in thesequence, one or more of the plurality of light sources are powered toenable illumination of the target area by powered ones of the pluralityof light sources. The method further includes capturing, from a singlereference location, an image of a target object that is located in thetarget area at each point in time in the sequence. The method furtherincludes calibrating one or more of the captured images includingbacking out discontinuities in lighting of the target area. The methodfurther includes evaluating the captured images including processing theevaluated images on a pixel by pixel basis to produce a single compositeimage having respective pixel values derived from the evaluated capturedimages. The method further includes providing, as an output, thecomposite image.

These and other implementations can each optionally include one or moreof the following features. Backing out discontinuities can includemultiplying each captured image by a normalization image based on thecorresponding calibration image. The normalization image can be createdby dividing each pixel value of the calibration image by the calibrationimage's maximum pixel value. The maximum pixel value can be evaluatedbased on luminosity.

In general, another innovative aspect of the subject matter described inthis specification can be implemented in a system comprising one or moreprocessors and one or more memory elements including instructions. Theinstructions can include providing an enclosure including a plurality oflight sources, each light source being disposed on the structure andbeing configured to illuminate a target area in proximity to thestructure, the enclosure including a frame and a screening cloak, thescreening cloak including a port for enabling capture of an image by acamera disposed on the structure at the port. The system can include acontroller for providing power and control for the plurality of lightsources. The system can further include an application for receiving anindication that a target item has been placed in the target area andthereafter controlling illumination of the plurality of light sources ina sequence over a first time period, wherein at a given point in time inthe sequence, one or more of the plurality of light sources are poweredto enable illumination of the target area by powered ones of theplurality of light sources. The application can further be configured tocapture, from a single reference location, an image of a target objectthat is located in the target area at each point in time in thesequence, evaluate the captured images including processing theevaluated images on a pixel by pixel basis to produce a single compositeimage having respective pixel values derived from the evaluated capturedimages and provide as an output, the composite image. The screeningcloak can be configured to extend over the frame to a base location soas to block ambient light from illuminating the target area. The framecan include adjustable height legs for adjusting a height of the portover the target area. The frame can be sized to receive a single page ofa photo album in the targeting area. The frame can be sized to receivean entire photo album but only illuminates and captures images one pageat a time. The application can execute in part on a mobile device thatis separate from the enclosure and controller. The screening cloak caninclude a pad for positioning the mobile device to enable image capturethrough the port.

Particular implementations may realize none, one or more of thefollowing advantages. A portable, low cost image capture system isproposed that allows for easy, quick and high quality image capture ofcontent. Image capture (such as for the purposes of digitization ofobjects, including albums, photos and books), can be done at home with auser's own cellphone based camera. Image capture can be completedwithout a user having to relinquish or mail off their objects orcontent. Users can receive and review a final product immediately andnot be required to wait substantial amounts of time between imagecapture and production of a final product. Costly commercial approaches,e.g., using large scanners and other complex technology, can be avoided.The equipment used for image capture can be extremely portable.

The details of one or more implementations of the subject matterdescribed in this specification are set forth in the accompanyingdrawings and the description below. Other features, aspects, andadvantages of the subject matter will become apparent from thedescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1C collectively show an example environment for two- andthree-dimensional image capturing, scanning and compositing.

FIG. 2 is a flowchart of an example process for 2D- and 3D-imagecapture, scanning and compositing.

FIG. 3 is a block diagram of an example computer system that can be usedto implement the methods, systems and processes described in thisdisclosure.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

Systems, methods, and computer program products are described for imagecapture and compositing. The quality of a finished product, such as animage, can depend on techniques that are used. For example, capturingcontent from photo albums and scrapbooks can be challenging because ofglare and page warping of the source material (e.g., pages/objects beingimaged). Traditional approaches may use, for example, large bookscanners (e.g., from commercial offerings) that address challengesmechanically, such as using a book cradle, a glass plate, and carefullight position. However, these approaches can be expensive andnon-portable.

In some implementations, techniques can use a portable, high-imagequality, low-cost image capture system. For example, the techniquesproposed can be used for media that are prone to lighting artifactshaving uneven lighting, reflections, and/or shadows. Media of this typecan include, for example, photo albums, scrapbooks, loose photos, framedimages, large artwork hanging on wall. Such techniques can be, forexample, part of a business model that allows image capture andprocessing without requiring a customer to part with their media,including providing an end product on the spot.

In some implementations, lighting can be provided and compensated for soas to minimize the effects of shadowing, glare or other light inducedartifacts. For example, in some implementations, for a given targetobject (e.g., a page of content from an album), an image can be capturedfor each light source (e.g., referred to as page capture). Each pageimage can be normalized against a reference image (e.g., an image of thetarget area without the content that can be used to set/adjust forlighting artifacts). A composite of the normalized images or portions ofthe normalized images can be created (e.g., referred to as a compositepage image). In some implementations, the compositing is performed pixelby pixel. Alternatively, surrounding pixels can be used in thecompositing. In some implementations, the compositing is performed usingless than all of the captured page images. Details of the image captureand processing are discussed in greater detail below.

FIGS. 1A-1C collectively show an example environment 100 for two- andthree-dimensional image capture and compositing. Referring to FIG. 1A,for example, scanning and compositing can be done using a structure 102and the structure's components, including a plurality of light sources104 mounted on the structure 102. Each light source 104, when aimed at atarget area 106, can be configured to illuminate the target area 106 inproximity to (e.g., below) a portion of the structure 102.

In some implementations, different approaches for lighting sources canbe used. For example, lighting sources can be directional and placed indifferent locations, such as evenly spaced around the perimeter of thearea to be imaged. Directionality of light can be maximized, forexample, using filters. With a larger number of effective lightingsources, lighting artifacts from any individual source can more easilybe removed in compositing, since artifacts from each image may comprisea smaller percentage of the combined image data. A single lightingdevice (e.g., LED, bulb, etc.) can produce multiple effective lightingsources. For example, multiple effective lighting sources can beproduced by physically moving lighting sources, by reflecting off one ormore mirrors (e.g., that are movable), or by using a combination oftechniques.

In some implementations, a configuration can include, for example, asingle LED centered above the scan area and pointed outward, reflectingoff a ring of mirrors surrounding the scan area. In someimplementations, a given number of lighting devices can also produce alarger number of effective lighting sources by using the lightingdevices in combinations. For example, combination can include using twoLEDs in two locations, the effective light from one LED or the otherLED, or the combination of both LEDs.

Referring again to FIG. 1A, a mobile device 108 (e.g., a mobile phonewith a camera) can be suspended by the structure 102 over the targetarea 106, such as with the camera lens is positioned/aimed at the targetarea 106. When in place, the mobile device 108 can capture images incoordination with illumination events provided by the light sources 104.The mobile device 108, for example, can be used for both imaging andcontrolling a controller associated with structure 102, e.g., by use ofa control box 112 (described below). For example, an application 113 onthe mobile device 108, while in communication with control box 112, canbe used to request image capture, perform image capture in coordinationwith illumination by the light sources 104, process captured images, andpresent composite images (or individual images) to the user, e.g., in auser interface (UI) 114. Image capture can occur, for example, wheninitiated by the user, such as after an object (e.g., a photo album 110)is placed in the target area 106.

As shown, eight light sources 104 are provided by the structure 102,e.g., along bottom edges of a top frame 116 of the structure 102,including along bottom edges of a cross piece 118. For example, thecross piece 118 can span a space between parallel long portions of thetop frame 116, such as along a boundary between a target half 120 and anon-target half 122 of the structure 102. As such, the light sources 104can be arranged in a rectangle, such as with light sources 104 at eachof the corners and substantially near the centers of the edges of therectangle. Other arrangements and numbers of light sources 104 arepossible. Although eight light sources 104 are shown in FIG. 1A, anysuitable number of light sources can be used (e.g., four or more).

In some implementations, image capture and compositing performed usingthe structure 102 can include coordinated actions performed by thecontrol box 112 and the mobile device 108. The mobile device 108, forexample, can capture images, e.g., when instructed by an application 113running on the mobile device 108. Similarly, the control box 112 can beinstructed by the application 113 on the mobile device as to the time,sequence of illumination events in order to capture each particularimage. While reference is made of communications in a certain directionand by instructions issued by the application 113, the intelligence forcontrolling illumination and image capture can be located elsewhere,such as in the control box 112. Communication 123 between the mobiledevice 108 and the control box 112 can be facilitated using Bluetoothcommunication or other techniques (e.g., near field communication (NFC),a wired connection, etc.). Coordination by the control box 112 andcommunication with the mobile device 108 can be timed, for example, tocoincide with the operation of the light sources 104. As such, themobile device 108 can be automatically instructed to capture pluralimages in rapid succession during a sequence of strobe flashes byvarious ones of the light sources 104.

The control box 112 can provide power to the components of the structure102, e.g., including providing power for the light sources 104. Forexample, one or more wires 124 can connect the control box 112 to thestructure 102, with wiring (not shown) running inside the structure andconnecting the light sources 104. The control box 112 can receive powerfrom an external power source, such as using direct current (e.g., usinga plug, not shown), batteries, solar, or other energy sources.

In some implementations, the mobile device 108 can be emulated (orequivalent capabilities achieved) using a mobile or non-mobile devicewith an imaging attachment that interfaces with the structure. Forexample, a USB attachment attached to a laptop computer and capable ofcapturing images can be used in place of the mobile device 108. Otherarchitectures are possible.

The control box 112 can control illumination of the target area usingthe plurality of light sources 104 such as in accordance with apredetermined, optimized, randomized or manual sequence over a firsttime period, such as in a time period of a few seconds. For example, ata given point in time in the sequence, one or more of light sources 104can be powered to light the target area 106. The sequence can include,for example, a series of light flashes from the light sources 104, theflashes timed to permit an image to be captured during a respectiveflash.

A single reference location 126 can be defined, e.g., in the center ofthe target area 106. From (and in accordance with) the single referencelocation 126, a plurality of images can be captured of a target object(e.g., a page of the photo album 110) located in the target area 106,e.g., at each point in time in the sequence. For example, a page of thephoto album 110 can be placed in the target area 106, such as slid orotherwise placed over the target area 106. The target area 106, forexample, can be a region of a substantially flat surface (e.g., a tabletop) on which the structure 102 stands.

An evaluation engine 115 (See FIG. 1C), for example included in theapplication 113 on the mobile device 108, can be used to evaluate thecaptured images. The evaluating can include processing the evaluatedimages on, for example, a pixel-by-pixel basis to produce one or more(e.g., a single) composite image having respective pixel values derivedfrom the evaluated captured images.

In some implementations, a 3D scanner 128 can be mounted on thestructure 102, such as along a bottom edge of the top frame 116 or alongthe bottom of the cross piece 118. The 3D scanner 128, for example, canperform 3D scanning of the target object to determine height informationfor a surface of a target in the target area for one or more pixelsassociated with the composite image. For example, the 3D scanner 128 canperform a 3D scan of a non-flat surface, such as a slightly bent and/oruneven page of the photo album 110. Uneven pages can also be caused byobjects placed on the pages, such as stickers or mementos. In someimplementations, more than one 3D scanner 128 can be used.

A compositing engine 117 (See FIG. 1C), for example included in theapplication 113 on the mobile device 108, can use the height informationobtained by one or more 3D scans to flatten respective pixels of thecomposite image. For example, captured images can be evaluated on apixel-by-pixel basis to produce a single composite image. The singlecomposite image can have respective pixel values derived from theevaluated captured images, as described above. In some implementations,various ways can be used to flatten a pixel value based on heightinformation from a 3D image. For example, a 3D scan can produce a pointcloud that defines a mesh surface. Mathematically the mesh surface canbe a Bezier surface in which each point in the 3D scan point cloud is acontrol point in the surface. The original non-flattened compositionimage can be mapped to this mesh surface. Then, the mesh can be deformedso that any control point whose height is non-zero is transformed tozero. The mapped image can then be projected back to two dimensions(e.g., since all heights are now zero, projecting to two dimensions isequivalent to just dropping the z dimension), and the flattened imagecan be provided. The generation of a mesh surface, the mapping of theimage to that mesh surface, the deformation of the mesh, and theprojection back to two dimensions are all commonly known mathematicalalgorithms and can be handled in software using existing transformationlibraries.

After the flattening of the respective pixels, the application 113 canprovide the composite image as an output. For example, the compositeimage can be displayed in the UI of the mobile device and/or provided asan output to another device/system (e.g., for storage on a USB device,for inclusion in an album, or for transfer to another party or system).

Referring to FIG. 1B, a cloak 130 can be provided to, for example, blockout ambient light. In some implementations, the cloak 130 substantiallycovers the entire structure 102. In some implementations the cloak 130only covers a portion of the structure 102, such as a portion associatedwith the target area 106. The cloak 130 can be used to cover thestructure 102 and underlying components and objects, for example, whenimage capture is to occur. The cloak 130 can prevent light from sourcesoutside the structure 102 from influencing image capture. The cloak 130can help ensure predictable lighting provided by the light sources 104during image capture. The cloak 130 can be made of cloth or othersuitable material(s) that provide easy and efficient covering of thestructure and object (e.g., the photo album 110) being imaged.

The cloak 130 can include a pad 132 upon which the mobile device 108 canrest, e.g., in a fixed position, during image capture. The pad 132 caninclude an opening 134, e.g., adjustable for (or customized to) aparticular device type of the mobile device 108. For example, thestructure 102 can include a pad 132 with an opening 134 that conforms tothe aperture location on the user's mobile device 108. In someimplementations, a saleable kit includes the structure 102, the controlbox 112 and an attachable (to the cloak 130) pad template that matchesthe device type specified by the user. Other manufacturing, shapesand/or configurations of the cloak 130 and its components are possible.

In some implementations, the opening 134 can be in the center of the pad132. In some implementations, the cloak 130 can includespacing/positioning mechanisms that help insure that the opening 134 isdirectly over the center of the target area 106, such as using loophooks or other fasteners to fix the cloak 130 to the top frame 116. Postprocessing of images can be used to account for irregular placement ofthe opening 134 relative to the target area 106.

In some implementations, during use, the cloak 130 can be entirely orpartially re-positioned for/after each imaging session. Therepositioning can include, for example, repositioning the cloak toenable moving a different object (e.g., another page of the photo album110, or some other object to be scanned) into the target area 106, thenreplacing the cloak. In some implementations, the cloak 130 can includea front flap or other portions that enable fast and simple access to thetarget area 106, enabling rapid re-deployment of the cloak 130.

Referring to FIG. 1C, the cloak 130 is shown in a completely deployedposition, e.g., shading outside lighting from sources outside thestructure 102. Further, the mobile device 108 is shown in position onthe cloak 130, e.g., with an aperture of the device's camera over theopening 134. In this position, for example, image capture can occur.

As an example, the user can initiate image capture using the userinterface. Upon determining that image capture is to begin, theapplication 113 can communicate with the control box 112. The controlbox 112 can send a series of commands to both the light sources 104 andthe mobile device 108 to capture plural images, using coordinated timingbetween the light sources 104 and the mobile device 108. Afterprocessing of the images (as described above), a resulting compositeimage 136 can be displayed in the UI 114 for presentation to the user.

In some implementations, image capture can be initiated in various ways,such as by voice commands or the use of a remote control or otherattachment that communicates with the mobile device 108 or the controlbox 112, or both. In some implementations, image capture can beinitiated upon user selection of a control in the UI 114, and after adelay of a few seconds (to allow any movement of the cloak 130 andpositioning of the camera), image capture can begin. Other ways ofinitiating image capture are possible.

FIG. 2 is a flowchart of an example process 200 for 2D- and 3D-imagingand compositing. In some implementations, components of the system 100can perform steps of the process 200 using instructions that areexecuted by one or more processors. FIGS. 1A-1C are used to provideexample structures for performing the steps of the process 200.

A structure is provided that includes a plurality of light sources(202). Each light source is disposed on the structure, and each lightsource is configured to illuminate a target area in proximity to thestructure. For example, the structure 102 includes light sources 104, asdescribed above, and each light source 104 is directed to the targetarea 106.

In some implementations, the plurality of light sources can include fouror more spatially separated light sources. For example, as shown in FIG.1A, there are eight light sources 104.

In some implementations, the structure is portable. For example, thestructure can be moved to any room of a home or workplace, can be easilycollapsed or disassembled and transported.

Illumination of the plurality of light sources is controlled in asequence over a first time period (204). For example, at a given pointin time in the sequence, one or more of the plurality of light sourcesare powered to enable illumination of the target area by powered ones ofthe plurality of light sources. For example, during image capture, thecontrol box 112 can instruct the mobile device 108 to capture imagescoincident with powering, in sequence, individual ones or groups of thelighting sources 104.

An image of a target object, located in the target area, is capturedfrom a single reference location at each point in time in the sequence(206). For example, the mobile device 108 can capture individual imagesrelative to the single reference location 126.

In some implementations, the process 200 further includes inserting apage into the target area, the page representing the target object. Forexample, a page can be positioned in the target area 106 for imaging.The page can be exposed for imaging, for example, by turning to theparticular page in the photo album 110.

The captured images are evaluated including processing the evaluatedimages on, for example, a pixel-by-pixel basis to produce a singlecomposite image (208). The single composite image has respective pixelvalues derived from the evaluated captured images. For example, thecomposite image 136, created by the compositing engine 117, can beproduced from single images of the photo album page in the target area106. In some implementations, evaluating the captured images can includeapplying a filter to each captured image.

In some implementations, a filter may be applied to the captured imagesprior to evaluation. For example, a Sobel filter may be used, whichtransforms the captured images to gradients which may be used insubsequent evaluation.

In some implementations, evaluating the captured images includesevaluating for each pixel of each captured image, determining one ormore pixel values from the captured images to use in the composite imageand determining a composite image pixel value for a respective pixelbased on the determined one or more pixel values. For example, theevaluation engine 115 can evaluate each of the captured images anddecide which pixels are to be included in the single composite image, asdescribed below. In some implementations, other techniques forcompositing multiple images can be used. For example, the gradient fieldcan be obtained from each image (such as by applying a Sobel filter), amedian of the input gradient fields can be determined, and the mediangradient field can be inverted back to the original pixel space, e.g.,using various numerical approaches. In some implementations, otherapproaches can be used, e.g., to transform each of the input images toanother domain, to apply a mathematical function to the set of images inthe secondary domain, and to transform the result back to the originaldomain.

In some implementations, evaluating includes for each pixel, filteringout a portion of the pixel values for one or more of the captured imagesand not using the filtered out portion in computing the determinedcomposite image pixel value. For example, filtering out a portion of thepixel values for one or more of the captured images can includesfiltering out one or more lightest and one or more darkest pixel values,as described below.

In some implementations, evaluating includes applying a mathematicalfunction to remaining pixel values from captured images not included inthe filtered out portion. For example, a mathematical function, such asa mathematical mean or median, can be applied to the remaining (e.g.,not filtered-out) pixel values to determine the composite image. In someimplementations, including the application of a mathematical functiondoes not require pre-filtering (e.g., pre-filtering can be optional).

In some implementations, evaluating includes applying a mathematicalfunction or algorithm against a plurality of pixels from each capturedimage to determine the value of each pixel in the composite image.

In some implementations, evaluating the captured images includesnormalizing each captured image with respect to the illuminationcalibration prior to further processing. For example, normalization ofan image can include compensation for uneven lighting and othernormalization techniques, as described below.

In some implementations, the target object is three-dimensionallyscanned to determine height information for a surface of a target on thetarget area for one or more pixels associated with the composite image(210). For example, the 3D scanner 128 can perform a 3D scan of theexposed page of the photo album 110.

In some implementations, 3D scanning includes determining a point cloudassociated with the target area including height information anddefining a Bezier curve to define a surface associated with the targetarea, and using the defined surface to develop the height information.For example, to define a Bezier curve for a particular 3D scan, a seriesof four distinct points can be converted to a cubic Bezier curve thatintersects the four distinct points. Other point cloud and curvetechniques can be used.

The height information is used to flatten respective pixels of thecomposite image (212). For example, the compositing engine 117 can usetechniques described below to flatten pixels in the composite image 136.

The composite image is provided as an output after the flattening of therespective pixels (214). For example, the composite image 136 can beprovided by the application 113 for display in the UI 114.

In some implementations, the process 200 further includes calibratingthe illumination including backing out discontinuities in lighting ofthe target area. For example, the application 113 can include amonitoring function that can make adjustments to the light sources 104to correct for recurring discontinuities in lighting over time. Othercalibrations can include, for example, changing the length of time thatlighting occurs for a particular light source 104, the aperture settingon the camera, and/or other calibration changes. In someimplementations, calibrating the illumination can include changing orrecommending a distance of the mobile device 108 to an object beingimaged. In some implementations, the structure 102 can includemechanisms for changing the height of an object placed in the targetarea 106, e.g., so that an exposed page of the photo album 110 isgenerally at a stable height relative to the camera.

In some implementations, each image can be normalized (e.g., prior tofurther processing) to compensate for uneven lighting (e.g., luxvariations across the image). For example, a reference image can becaptured of a uniform background (e.g., planar, single color, singletexture). The page image can be captured using various captureparameters, e.g., exposure, shutter speed, aperture setting, and whitebalance. If lighting is even, for example, then the reference imageshould have a same value for all pixels.

In some implementations, to compensate for uneven lighting, thefollowing procedure can be used. A 2×2 normalization matrix is createdbased on the reference image, where the matrix width and height are thesame as the reference image. The maximum pixel value of the referenceimage is determined (i.e., the value of the pixel from the referenceimage with the maximum value based on one of the evaluation methodsdescribed below). The value of each cell of the normalization matrix isthe value of the corresponding pixel from the reference image divided bythe maximum pixel value previously determined. Finally, uneven lightingin the captured image can be compensated for by multiplying pixel-wisethe captured image by the normalization matrix. The resulting image isreferred to as the normalized image. The maximum pixel value, forexample, can be dependent on a mean maximum lightness, e.g., the L inhue, saturation, light (HSL). In a red, green, blue (RGB)representation, assuming the reference image is white, the pixel can beselected based on the maximum value of any single channel (R, G, or B).Thereafter, the system can use the values of all channels for theselected pixel. This technique can be used, for example, instead offinding the maximum value of each channel independently, which canresult in a color shift in the normalized image. Finally, each pagecapture can be multiplied by the normalization image to create thenormalized page capture.

In some implementations, creating the composite of the normalized imagescan include using composite multiple images to remove directionallighting artifacts (e.g., glare and shadows). Various approaches ofcompositing can be used. In a simple median approach, for example, twomiddle images can be averaged for an even number of images. In adiscarded-outliers approach, for example, outliers can be discarded, andthe remaining images can be averaged. For example, for each pixel,values can be sorted, and the top 25% and bottom 25% can be discarded,and the remaining 50% averaged. In another approach, for example,outliers can be discarded based on a fixed maximum value difference fromthe average or based on a fixed maximum standard deviation from theaverage. In another approach, for example, glare/shadows can be detectedbased on differences from surrounding pixels, as compared to that inother images. For these compositing approaches and/or others, sortingcan be based on, for example, the L value in HSL, or a single componentvalue in RGB (e.g., rather than sorting each channel separately). Insome implementations, an operator (or an end user) can influence thecompositing process by explicitly identifying areas affected byglare/shadows. This can be useful, for example, if there are not enoughlighting sources to remove all lighting artifacts.

In some implementations, de-warping techniques can be used incompositing images. For example, a page can be 3D-scanned and de-warpedin software, given that page can be expected to be primarily planar.Exceptions can occur, e.g., for objects placed on the page, such askeepsakes or 3-dimensional objects (e.g., stickers). In someimplementations, de-warping can be a separate, optional step after imagecompositing.

In some implementations, various scanning approaches (e.g., associatedwith de-warping) can include the use of stereo imaging (e.g., optionallywith an infrared (IR) dot pattern) and the use of a positionable linelaser. In some implementations, using the positionable line laser canuse, for example, a rotary barcode scanner (e.g., using rotatingmirrors) or a laser on a servo. Calibration of the positionable linelaser, for example, can use an object of known size, so that systemdoesn't depend on an exact position/angle of the laser. Calibration canalso be used, for example, to determine actual pixels per inch (PPI) andstore in final image exchangeable image file format (EXIF) information.

Scanning using a positionable line laser can include various actions.For example, images of a line projected across a scan surface can becaptured at various positions (e.g., with more distinct positionsyielding higher resolution scans). During calibration, for example, foreach scan, at least one side of the line can be required to project ontothe scan background (e.g., not the page being scanned). Duringcalibration, based on a known height of a calibration object and basedon a difference in the position between the line projected on acalibration object and the line projected on the background, the planedefined by the laser projection can be determined. The determination canassume, for example, that the laser projection rotates around an axis;then given two images (with distinct laser positions), the line on whichthe laser resides can be determined. To 3D-scan a page, for example, oneimage can be captured for each calibration image (e.g., with matchinglaser positions). Matching laser positions can be determined, forexample, either by a known servo position (in the case of controllableposition) or by ensuring that the projected line reaches the scanbackground (on at least one side of the image). The calibration imagecan be determined with matching line position. For each scan image,given the known laser position and angle, for example, the height of thepage can be determined at each point along the laser projection. In someimplementations, a 3D scan image need not be the same resolution as theimage scans, as differences can be accommodated by the de-warpingprocess.

In some implementations, de-warping can include the followingtechniques. For example, using the 3D scan point cloud, a Bezier surfacecan be determined through the control points. For each pixel in the pagescan, an affine transform can be performed on the pixel from theposition mapped on the Bezier surface to the origin plane (e.g., samex/y coordinates, but z=0). In some implementations, to accommodateactual 3D objects on a page (e.g., raised stickers, keepsakes, etc.)versus curvature of the page itself, the 3D objects can be identified.The affine transform can then be adjusted accordingly (e.g., bytransforming to the height of the object instead of to z=0). 3D objectscan be identified, for example, by performing edge detection on animage. In some implementations, normal edge detection (e.g., based on agradient filter) can be performed, and 3D objects can be identifiedbased on edges coinciding with height changes from the 3D scan. In someimplementations, edge detection sensitivity can optionally be controlledby an operator or an end user.

In some implementations, a 3D model and known lighting positions can beused to further normalize lighting. For example, reference imagelighting may differ from page lighting, depending on light sourceposition and page elevation. Normalizing page image based on referenceimage can be inaccurate, e.g., when the 3D surface of the page differssignificantly in height or angle from that of the reference imagebackground. The 3D model can compensate for these differences, since theexpected brightness can be determined based on the surface angle and thedistance from the light source, which can be inferred from the 3D model.

In some implementations, a remote trigger device can be used to avoidmoving the camera (e.g. that is part of a mobile device) during imaging.For example, the remote trigger device can communicate with the mobiledevice either directly or through a control box.

In some implementations, automatic cropping can occur, e.g., based on anexpectation that a page is rectangular, usually with a spine on oneside, and captured against a known background. For example, automaticcropping can use edge detection on the image scan or a 3D scan, or useboth sources to improve accuracy.

In some implementations, an application on a mobile device can be usedto capture images from pages in an order that allows for fasterprocessing of multiple scans. For example, the application can bedesigned to be able to capture even pages, then odd pages, and sort thecaptured pages for processing and/or presentation.

In some implementations, the application can support integrated billing.For example, scans can be billed automatically to a credit card, a useraccount, or some other billing/payment system.

In some implementations, the application can support integrated outputof captured and composited images to physical storage (e.g., USB sticks,etc.), uploading to the cloud for viewing, sharing and customization,and/or allow a customer to use raw images to customize compositing andde-warping.

In some implementations, other processes can be used, including anadditional process that is different from the process 200 for 2D- and3D-imaging and compositing. In some implementations, components of thesystem 100 can perform steps of the additional process usinginstructions that are executed by one or more processors. In someimplementations, FIGS. 1A-1C are used to provide example structures forperforming the steps of the additional process. In some implementations,an additional system described below can be used for performing thefollowing steps of the additional process.

A structure is provided that includes a plurality of light sources. Forexample, each light source can be disposed on the structure and can beconfigured to illuminate a target area in proximity to the structure.

One or more calibration images of the target area are captured. Forexample, the capturing can include illuminating the target area with oneor more of the plurality of light sources and capturing the one or morecalibration images coincident with the illuminating.

An indication is received that a target item has been placed in thetarget area. Thereafter, illumination of the plurality of light sourcescan be controlled in a sequence over a first time period. For example,at a given point in time in the sequence, one or more of the pluralityof light sources can be powered to enable illumination of the targetarea by powered ones of the plurality of light sources.

An image of a target object is captured from a single referencelocation. For example, the image can be located in the target area ateach point in time in the sequence.

One or more of the captured images are calibrated including backing outdiscontinuities in lighting of the target area. In some implementations,backing out discontinuities can include multiplying each captured imageby a normalization image based on the corresponding calibration image.For example, the normalization image can be created by dividing eachpixel value of the calibration image by the calibration image's maximumpixel value. In some implementations, the maximum pixel value can beevaluated based on luminosity.

The captured images are evaluated including processing the evaluatedimages on a pixel by pixel basis to produce a single composite image.For example, the single composite image can have respective pixel valuesderived from the evaluated captured images. The composite image isprovided as an output.

In some implementations, the additional process can be implemented inthe following system. For example, the system can provide an enclosureincluding a plurality of light sources, each light source being disposedon the structure and being configured to illuminate a target area inproximity to the structure. The enclosure can include a frame and ascreening cloak, the screening cloak including a port for enablingcapture of an image by a camera disposed on the structure at the port.

The system can include a controller for providing power and control forthe plurality of light sources. The system can include an applicationfor receiving an indication that a target item has been placed in thetarget area and thereafter controlling illumination of the plurality oflight sources in a sequence over a first time period. For example, at agiven point in time in the sequence, one or more of the plurality oflight sources can be powered to enable illumination of the target areaby powered ones of the plurality of light sources. The application canbe further configured to capture, from a single reference location, animage of a target object that is located in the target area at eachpoint in time in the sequence, evaluate the captured images includingprocessing the evaluated images on a pixel by pixel basis to produce asingle composite image having respective pixel values derived from theevaluated captured images and provide as an output, the composite image.

In some implementations, the screening cloak can include, for example, apad for positioning the mobile device to enable image capture throughthe port. In some implementations, the screening cloak can be configuredto extend over the frame to a base location so as to block ambient lightfrom illuminating the target area. In some implementations, the framecan include adjustable height legs for adjusting a height of the portover the target area. In some implementations, the frame can be sized toreceive a single page of a photo album in the targeting area. In someimplementations, the frame can be sized to receive an entire photo albumbut only illuminates and captures images one page at a time. In someimplementations, the application executes in part on a mobile devicethat is separate from the enclosure and controller.

FIG. 3 is a block diagram of example computing device 350 that may beused to implement the systems and methods described in this document, aseither a client or as a server or plurality of servers. Computing device350 is intended to represent various forms of mobile devices, such aspersonal digital assistants, cellular telephones, smartphones, and othercomputing devices. The components shown here, their connections andrelationships, and their functions, are meant to be examples only, andare not meant to limit implementations of the inventions describedand/or claimed in this document.

Computing device 350 includes a processor 352, memory 364, aninput/output device such as a display 354, a communication interface366, and a transceiver 368, among other components. The computing device350 may also be provided with a storage device, such as a micro-drive orother device, to provide additional storage. Each of the components 350,352, 364, 354, 366, and 368, are interconnected using various buses, andseveral of the components may be mounted on a common motherboard or inother manners as appropriate.

The processor 352 can process instructions for execution within thecomputing device 350, including instructions stored in the memory 364.The processor may also include separate analog and digital processors.The processor may provide, for example, for coordination of the othercomponents of the computing device 350, such as control of userinterfaces, applications run by computing device 350, and wirelesscommunication by computing device 350.

Processor 352 may communicate with a user through control interface 358and display interface 356 coupled to a display 354. The display 354 maybe, for example, a TFT LCD display or an OLED display, or otherappropriate display technology. The display interface 356 may compriseappropriate circuitry for driving the display 354 to present graphicaland other information to a user. The control interface 358 may receivecommands from a user and convert them for submission to the processor352. In addition, an external interface 362 may be provided incommunication with processor 352, so as to enable near areacommunication of computing device 350 with other devices. Externalinterface 362 may provide, for example, for wired communication (e.g.,via a docking procedure) or for wireless communication (e.g., viaBluetooth® or other such technologies).

The memory 364 stores information within the computing device 350. Inone implementation, the memory 364 is a computer-readable medium. In oneimplementation, the memory 364 is a volatile memory unit or units. Inanother implementation, the memory 364 is a non-volatile memory unit orunits. Expansion memory 374 may also be provided and connected tocomputing device 350 through expansion interface 372, which may include,for example, a subscriber identification module (SIM) card interface.Such expansion memory 374 may provide extra storage space for computingdevice 350, or may also store applications or other information forcomputing device 350. Specifically, expansion memory 374 may includeinstructions to carry out or supplement the processes described above,and may include secure information also. Thus, for example, expansionmemory 374 may be provided as a security module for computing device350, and may be programmed with instructions that permit secure use ofcomputing device 350. In addition, secure applications may be providedvia the SIM cards, along with additional information, such as placingidentifying information on the SIM card in a non-hackable manner.

The memory may include for example, flash memory and/or MRAM memory, asdiscussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 364, expansionmemory 374, or memory on processor 352.

Computing device 350 may communicate wirelessly through communicationinterface 366, which may include digital signal processing circuitrywhere necessary. Communication interface 366 may provide forcommunications under various modes or protocols, such as GSM voicecalls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, orGPRS, among others. Such communication may occur, for example, throughtransceiver 368 (e.g., a radio-frequency transceiver). In addition,short-range communication may occur, such as using a Bluetooth®, WiFi,or other such transceiver (not shown). In addition, GPS receiver module370 may provide additional wireless data to computing device 350, whichmay be used as appropriate by applications running on computing device350.

Computing device 350 may also communicate audibly using audio codec 360,which may receive spoken information from a user and convert it tousable digital information. Audio codec 360 may likewise generateaudible sound for a user, such as through a speaker, e.g., in a handsetof computing device 350. Such sound may include sound from voicetelephone calls, may include recorded sound (e.g., voice messages, musicfiles, etc.) and may also include sound generated by applicationsoperating on computing device 350.

The computing device 350 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as asmartphone 382, personal digital assistant, or other mobile device.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. Other programming paradigms can be used, e.g., functionalprogramming, logical programming, or other programming. As used herein,the terms “machine-readable medium” “computer-readable medium” refers toany computer program product, apparatus and/or device (e.g., magneticdiscs, optical disks, memory, Programmable Logic Devices (PLDs)) used toprovide machine instructions and/or data to a programmable processor,including a machine-readable medium that receives machine instructionsas a machine-readable signal. The term “machine-readable signal” refersto any signal used to provide machine instructions and/or data to aprogrammable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device(e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor)for displaying information to the user and a keyboard and a pointingdevice (e.g., a mouse or a trackball) by which the user can provideinput to the computer. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback (e.g., visual feedback,auditory feedback, or tactile feedback); and input from the user can bereceived in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here), or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular implementations of particularinventions. Certain features that are described in this specification inthe context of separate implementations can also be implemented incombination in a single implementation. Conversely, various featuresthat are described in the context of a single implementation can also beimplemented in multiple implementations separately or in any suitablesub-combination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asub-combination or variation of a sub-combination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

Thus, particular implementations of the subject matter have beendescribed. Other implementations are within the scope of the followingclaims. In some cases, the actions recited in the claims can beperformed in a different order and still achieve desirable results. Inaddition, the processes depicted in the accompanying figures do notnecessarily require the particular order shown, or sequential order, toachieve desirable results. In certain implementations, multitasking andparallel processing may be advantageous.

What is claimed is:
 1. A method comprising: illuminating a target areausing a plurality of physical light sources, wherein each light sourceis positioned on a physical structure relative to the target area andpositioned to illuminate at least a portion of the target area whenactivated, and wherein the illuminating comprises controllingillumination of the plurality of light sources in a sequence over afirst time period, wherein at each of a plurality of points in thesequence during the first time period a different combination of one ormore of the plurality of light sources are powered on to provideillumination of the target area; capturing, using a mobile devicedistinct from the physical structure, a plurality of images of thetarget area over the first time period representing different points inthe sequence, wherein each image captures a target object that islocated in the target area according to the illumination provided bythat point in the sequence; evaluating the captured images includingprocessing the captured images to generate a single composite image; andproviding the composite image to a server system.
 2. The method of claim1, wherein capturing the plurality of images comprises: sending one ormore commands to a control unit of the physical structure, the one ormore commands specifying the sequence of illumination over the firsttime period.
 3. The method of claim 1, wherein capturing the pluralityof images comprises: providing instructions to the mobile device, from acontrol unit of the physical structure, to capture the plurality ofimages.
 4. The method of claim 1, wherein the sequence of theillumination of the plurality of light sources is according to apredetermined sequence.
 5. The method of claim 1, wherein capturing theplurality of images comprises: using a control unit of the physicalstructure to send commands to both the plurality of physical lightsources and the mobile device to capture the plurality of images usingcoordinated timing between the light sources and the mobile device. 6.The method of claim 1, further comprising capturing one or more imagesusing a second image capture device to generate one or more stereoimages of the target area, wherein the one or more stereo images areused for de-warping during generation of the composite image.
 7. Themethod of claim 1, wherein the composite image is generated based onpixel-by-pixel processing the plurality of images.
 8. A methodcomprising: illuminating a target area using a plurality of physicallight sources, wherein each light source is positioned on a physicalstructure relative to the target area and positioned to illuminate atleast a portion of the target area when activated, and wherein theilluminating comprises controlling illumination of the plurality oflight sources in a sequence over a first time period, wherein at each ofa plurality of points in the sequence during the first time period adifferent combination of one or more of the plurality of light sourcesare powered on to provide illumination of the target area; capturing,using a mobile device distinct from the physical structure, a pluralityof images of the target area over the first time period representingdifferent points in the sequence, wherein each image captures a targetobject that is located in the target area according to the illuminationprovided by that point in the sequence; and providing the plurality ofimages to a server system for processing.
 9. The method of claim 8,wherein capturing the plurality of images comprises: using a controlunit of the physical structure to send commands to both the plurality ofphysical light sources and the mobile device to capture the plurality ofimages using coordinating timing between the light sources and themobile device.
 10. The method of claim 8, comprising: calibrating theillumination provided by the plurality of physical light sources. 11.The method of claim 10, wherein the calibration includes backing outdiscontinuities in lighting of the target area.
 12. The method of claim10, wherein the calibration includes changing a length of time thatlighting occurs for a particular physical light source.
 13. The methodof claim 10, wherein the calibration includes monitoring illuminationand making adjustments to the physical light sources to correct forrecurring discontinuities in lighting over time.
 14. An apparatuscomprising: a physical structure that includes a plurality of physicallight sources, wherein each light source is positioned on the physicalstructure relative to a target area and positioned to illuminate atleast a portion of the target area when activated; a control devicecomprising one or more processors and one or more memory devices, thecontrol device configured to control the physical structure in responseto instructions received by a remote device distinct from the controldevice, wherein controlling the physical structure comprises; responsiveto the received instructions from the remote device, illuminating thetarget area using the plurality of light sources, wherein theilluminating comprises controlling illumination of the plurality oflight sources of the physical structure in a sequence over a first timeperiod, wherein at each of a plurality of points in the sequence duringthe first time period a different combination of one or more of theplurality of light sources are powered on to provide illumination of thetarget area.
 15. The apparatus of claim 14, wherein one or more of theplurality of light sources are movable to increase a number of effectivelighting sources for illumination of the target area.
 16. The apparatusof claim 14, wherein the physical structure includes a movable targetarea so that the height of the target area is adjustable.
 17. Theapparatus of claim 14, wherein the physical structure includes an imagecapturing device that can capture images in combination with the remotedevice to generate stereo images.
 18. A method comprising: receivinginstructions, at a control device of a physical structure having aplurality of physical lights sources, from a remote device to illuminatea target area relative to the plurality of physical light sources; andresponsive to the instructions received from the remote device,illuminating the target area using the plurality of physical lightsources, wherein each light source is positioned on the physicalstructure relative to the target area and positioned to illuminate atleast a portion of the target area when activated, and wherein theilluminating comprises controlling illumination of the plurality oflight sources in a sequence over a first time period, wherein at each ofa plurality of points in the sequence during the first time period adifferent combination of one or more of the plurality of light sourcesare powered on to provide illumination of the target area.
 19. Themethod of claim 18, comprising: calibrating the illumination provided bythe plurality of physical light sources to the target area.
 20. Themethod of claim 18, comprising: using the control unit of the physicalstructure to send commands to the remote device to capture the pluralityof images using coordinated timing between the sequence illuminating thelight sources and image capture by the remote device.